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Linear Dynamic Recursive Estimation from the 
Viewpoint o f Regression Am. lysis 

D, B. DUNCAN and S, D. HORN* 


AL'g Jo c -='J *« c jc«rsivj> f6*e*«5«*m«Stoi* onrf -r-.it n 

■ h.'l 11 . - ' t m ••» ! D. r • '1 tt o-.d 

utiSty wWc i m - ■ h - ' *> « ' t 1 * 

a • , > - < , - i - « 3 * l T°« 

„ f a ] p t , . , . - u « »cV.- e* 

teruion of conv«r.t»f. J ft«KS-,J %,re„on !heo<-> and 3 r« the cp mat 
recsirrive asWmotofS «i terms of the extended regression theory far a t,pkn! 
form of the r«u-sivc mcde!. Th» ol» opeM the way fo-- further developments 
,-,„», i a ~ a -f< ' » PP'O 

and wii! be presented Is Mure sopors. 

]. INTRODUCTION 

A v.ry u-eful ehs* o* multivariate time series model* 

>nd -t.ti, * 1 '1 , > J < ! 1 

I »! u J HI [" 1 *' ' ' ' 

t tie* ,u<h f ',' i 7 arirf precis 

fi'on. Th. * ar n-.'f<b for all --orK ,f p.-ohle wr>ii,- 

fr mi i . . { 'i i •,.;>!<--*•• > dp 1 

dii ti the . . ii' < • ■ .1 ilh < i i 1 t ' ' r 

Devp j h.-k «S( - .-•>,!!!,. V If- t M?l« r 

cal regression theor\ an.', d^spih ife- hie;li utility f r pro- 

iding 'it uf» th e ibfTi t cvteit-ioi 

regres-'i problems ifie •' * i- > > >- ia- im? t the 
stt< i i nnft ,n ^ -i t * i i, . rl i „ ■ 1 ! g h 
theon Wt feel th - lack of , .; > nu- . itio; di . to th^ 
favi *l> u the - t the, i has not i.-e. . .> ! - r d it £ 
direct terisi. i: of res;r> ss on thenn but inst< ad from a 
alff' t nt though, eg i.t point »f i w aide sense con- 
ditional P i babtlit th, .ft e.g 3j) 

1?' an i f! .it to ver< ona th'-s < ommim cations b!o< k we 
•now how a typical form of linear dvnamie recursive esti- 
mate n rat h < > • a rut. 1 i .t :r 
sion theor J' d< tin u n.ii.M re.-fiu rir.-t i wit m 
rand m-3 regressi nth in 01 I rabl nt< stir ts 
own r 1 - < h 1 h h ; 1 pr msl 
in the statistical literature (e.g.. [2]. [4], [/]. [9]). 

Ih , ( > mo ie! 1 it. tcr 

°f hi ecurai\ < • s. A typical u n for a rariat 
time st•r;e^ >'., ■ ■ . y~ conM.ste of (It an observation 
equation: y t =X,2 t +€, for times and (2) a 

dynamic regression oefficients transitio; equation: 3» 
= T,J ( .„j+u ( . i=2, • ■ • , f<. starcuiK with = at 

* £* » , . ~ , . i - >■ ' < i a . u « i. 


;-=l\iW';..bak prior nvau for 5i Fa^h observa- 
tion equator; of the fam Lis r regr. »sion form : which 
the regression coefficient vector ?,('•'<!) it- c«lh'd the 
tcj.- c t,.f t X t 'p/n\>, . kn. -no.tr, of n >rr.— or=t, 
aial E-'pXl, i- a vector of random errors. 

rht d\rt .in t f f i - . ti..! =T 1 'If! 
s ne « to i . t t d tise< ir « n heorj twl ex- 
presses the State J< as a kn«v » linear tra - -mutton 
7, Kr) f rh previ i= tal }, plut-i • 'Tu.frXl) 
of ra J . • rrurs 

Th-, luteal operation on 3<_i and the introduction of 
in .- r- what ... ■ v tts ft' "3 'f>'«« • i-pe't 
\ situation .v 3 r f . tpatti. win. eg n u. the 
vjectu; t | *i 4 1 n.adlj a- any 
f it t It 1 1 ptt i tie ter-* describing 

the trajectory at time I depend on those at time <• - 1 

Tim dynarn i it it< tr; ti ,n equatn the starting 
u , ^ v -<f tl - mipt > t - il ut ti :rr - 
vect-w u ( . /=- 1, • • • , « provide a prior model for the 
.e<&ifie(fr v«StQ? gu>f.3ill tike jre^Sdiotx efee^ients or state 
elements concerned, 

Is i nventi ' reg e > the ry, th< m >u> • 5 
made about the error vector? e ( and u. are of one of two 
f nnf I r h < r t ' - f< , t- an i c^untpttou 
Hth , vectors have, zero expectations all ar ui r 
related with . v an ther and -o' hav< knowi i triance 
covarian e matrix Q var(u t ) and R t = var oi 11/ 
Fa h ern.r vav' h hs - th ran tivar iti norm 1 (oi sian) 
di?tribution, a t ^N(Q, Q,} and z,~*N(0, R t ). In the 
\ ..a!; r form, the ; i enn a mmpt tr i dt tribu- 

m , to, t ad • . t the -'an i s specif*, 
intr, the first and second moments. We denote the^e by 
u,~~WS(0, 5uo. : -;^0«. 'i (lanssiancase, 
of course, all t! error \ i < at 1 ' ete"t 

The follow it g -pe. ial : t ous r-du<- A.- new dynamic 
m -del to the convent, i ixed-3 regression model 
y = A'3- i- 1 The state transition equation J, = T,3 , 
+ u, is now- the trivial identity transformation (i.e., 
7\ = /„nd u,=O<-»0 t = O; and ?i is fixed and unknown. 
Tins is the otih % »ct. , ..; t -gr. - on coeffit ents involved, 
st. th. need for the subs< ipt is removt < Ina- \<- m^y 

i i i ~ i ifn till 2 [lie thobsen ttit jtuiti i 
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y, =X ( ? ( + t l is univariate p — 1 and represents the tth 
row y t = x!§+tt of the conventional model y=X?-f-t 
where J/,(IX1) is the tth element of y, xl -X,(l Xr) is 
the tth row of X", etc. 

The most important advantage of the new work is the 
increased versatility of the dynamic model, i.e., the en- 
larged range of real and important problems involving 
univariate or multivariate (p>l) non-stationary time 
series with which it can cope. 

Another advantage is the simplicity of the recursive 
methods for the efficient handling of the optimal estima- 
tion of extremely large numbers of parameters. A se- 
quence of ver\ - n ; > h ition replaces the 
direct simultaneous solution for all of the ? parameters 
involved. In a typical form of these methods, filtering, an 
optimal estimator b t of ?, is obtained recursively as a 
linear combi * of the j t ttor k n 
y t , the response vector at time t. The recursive filtering 
equations simultaneously provide the variance-cov&ri- 
ance matrix S, = var(b,-?,) of the estimator. Both equa- 
tions hi\( 1 * i in an iipdat , form which is 
useful especially in programmed computing (cf. (3.5a) 
and (3.5b)). 

It is well known from the engineering literature that 
under the wide sense error assumptions the filter 
r b t foi * rt i) se of being the 

mum ir 

«fi, yi, 
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square linear estimate 
, y t up through time 


r based on all the data 
Under the Gaussian 


< the 


volved. In regression terms, this is "fitting a regressMi^f 
§t on jii, yi, ■ • , y." The normal equations for doing 
this are based on the expected first and second mumentg 
of the variables involved. 

The purpose of the present paper is to overcome the 
communications block by: (1) explaining the new dy- 
namic models and methods in the terms and notation of 
conventional regression theory, and (2) deriving the 
optimally of the recursive estimators in terms of * 
natural random-? extension of the conventional fixed-} 
regression theory. 

The rand.-r gn tl hould servt 

as a two-way communications bridge; a bridge by which 
the conventional regressionist can move to an easier 
understanding of the more direct derivation of recursive 
estimators, and by which the engineer can move to an 
easier understanding of the relevance to his problewflfc 
many valuable results in regression theory. ApartigR 
these objectives, the random-? extensions of fixed$4t 
•gression theory, e.g., the extended Gauss-Markov Jp 
orem, are of considerable interest in themselves, and of 
potential value for a wide class of applications. f? 

2. A WIDE-SENSE RANDOM ? REGRESSION THEORY : 

In this section we extend well-known definitions and 
theorems of fixed-? regression theory in a natural way to 
form a random-? regression theory required for a simple 
regression development of the Kalman recursive equa- 


spee 


ns of the updating equations give similarly opti- 
nators b,,, oi § I on all the data y • ■ , 
rough time s, for in > g ike f , .! < Of 
iterest are the estimators l>»j, which are optimal 
i all the data. 

D i ife its . t i natural exl nsion of con- 

ventional r,-gr - ,nth ry, th g > i ng approach has 
so far been lit " -d or used by the conventional 
regression analyst, perhaps due partly to the unfamiliar- 
ity of the notation and terms in the engineering literature. 
But more importantly, the neglect is due to a confusion 
caused by the use. in its development, of a theory in 
which the * .i .e- ir tl me in t n)o vf ntu n il r t cr - 
sion theory but the roles are different. Specifically, the 


e.lral objective in both 


2.1 The Model 

The model can be written in the familiar fixed-? regres- 


, th t } m v\ 

similar style as 


t where t~WS(0, S) • (2.1) 
s random and may be written in a 


where 


- WS(0, Xj) (2.2) 


used Hi 

odel that 


orrelated with t. The notation z~~WS(\f, X), 
the introduction, indicates the wide sense 
: has any distribution with mean * and vari- 


late 3, the entire 


e.'liei* 


e.I. 


• J. s 


" 3 a 


t.ht;. 


od, I 


i the 


equation for y. T> h i. n ,e „• - . < .1 f 

i ;h! ...ikf ~> i. t I e ' i t t |. i > » 1 J&odom van- 
able. By a suitable redefinition of the sector space terms 
involved, the optimal estimate itself is found directly as 
m orthogoual projection of % on a vector space 
spanned h\ the vectors ui, y h ■ • - , y, of the data in- 


« is an / XI known prwi nu 
a is an /XI vector of prior 
X(nXn) and SjfrXr) sre v 
are known except pnwib! 

i Model. I 


(2.3) 
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for 


the whole model, (2.1) and (2,2) above, may be written. 
ii< ' 1 rufl.v as 


-IT 'J 


with dimensions y& XI). XmXr). £:»X«), »-r+«. 

It is precisely this condensation that allows us to ex- 
tend the fan 1 j. i i' '» ch -iinL-ni 
3 case with a minimum of new techniques needed to 
prove the required results. 

In fixed-g t : mpor t -tin u r is the WLSB 

tc.jh'.t ! ' it of The natural ex- 

tension of trn^ d 

Den>,ii>,/. 2-:: U't >,'• ' L» i > /xares Estimator. The 
vector fe(rXl) is the W.LSE of g if 


s minimized when 3 = b. 


XV; 


(2.6) 


X u I BE of g is 
given by the solution of the Normal Equations 


(2.7) 


Proof.- The quadratic form (2.6) to be minimized has 
the same form as that, in fixed-g theory where it is known 
that the minimum is given by the solution of (2.7). 

By st i i tor b g > i 2 7) is seen 

to be the familiar weighted combination: 


V = IXj- 


vlsr'» + X'ST'y] 

x'x-^x]-' 


Lemma $.4: Wide-Sens 
that 


Distribution 

(j - xgj ~ ws% i) 


b-Q. Given 


(1) (b - 

(2) £(b 

(3) i?(b 


- g)(y - Xfc)' = 0, and 

- g)y' = o. 


Proo/: We Erst note that 
(2 ' 4) (b - 3) = (X'S-XJ-'X'S-'fi - Xg) 

= (X'i-'xy-^xs-'Ir 

(2 .5) (y - X6) = [I - X(X'£ - 'Xj-'X'S-'jy 

- [/ - X(X'S-'X)- ! X'S-' l(Xg + ~e) (2. 1 1 5 

- [/ - x(X'i- l x)- , x' 

Results (1) and (2) follow directly. 
From (2.1) and (2.2) 

j- = X3+ £ =X|t + Xu + E . (2.12) 

From (2.10) and (2.12) we get 
£(6-3)y'-(X'£- l X)~ l X'X- ! 

{[""]»'*'+ [~"](«'X'+c')} 

=o+(X'i t x)-H"^- 1 X3X'+x'i;- ! £) = o. . 

Since £0>-g)t»'=--0 also, result (3) follows. 


2 10) 


(2.8) 
(2.9) 


Definitions B.o; An estimator y(gXl) of a!1 >' « X1 
linear vector function y = Cg of g is 

1. linear if it is linear in y, i.e., if f =-*r -Xiif + Atr, where 
A, A s and -4 S are ? X 5,?Xr and qXn respectively; 

2. unconditionally unbiased (u-unbiased) if E(r-y)>=0: 
i square if Eh — y<)' is minimized for each 

(?. — -r>) is minimized for each 
i-l, ■ • - 

5. tte MVLUE (minimum variance linear «-unbiased es- 
timator) if for every yi no other estimator has equal or 
smaller variance in the class of all linear u-uiibiased e»- 


4. minimum variance if v 


The distribution of the WLSE b of 3 m wid- -> 
fixed-g theory is known to be b~WS(&{2CXr*Xr 1 )- 
Note that it is this fixed-? estimator weighted inversely 
by its variance which gives the X'X~ l y term in (2.8). 
Intuitively, in the random-g case interest is centered on 
the distribution of 6— 3, since this difference tells how- 
well a random (5 is being estimated. 


„, .-sofwi-l, 

6. the MMSLS (minimum mean square linear estimator) 
if for every yi no other estimator has equal or smaller 
mean square in the class of all linear estimators of y it 
•••«?- 

Because a linear estimator y is linear in jt as well as y 
we get the following useful result in the random-g case. 

Lemma £.6: Equivalence of the MVLUE and the 
MMSLE. Given that (?-Xff)~*FS(0, X), the MVLUE 
of any linear vector function y = Cg of g is the MMSLE 
of y. 

Proof: Suppose »*0. Let y be linear, 
4-Asy, and consider 

m, - v.) 1 = [Wt - ^.)l 2 + «*W< - 5 x 

fori = 1, 


(2.13) 


b « (X'S-'Xj-'X'S-'y 


Since Ai and » are non-random, the var(f<— y<) does not 
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depend on Ai (or for i = 1, ■ • 
the right-hand side of (2.13) in two steps: 

1. Choose At to minimize var (7, — >») for t = 

2. choose A, (which will be a function of <4i) 
the bias E(y>-y,) = 0 for i = 


From Step 2 it follows that the MMSLE is w-unbiased 
and thus the M VI 1 t! MMSLE of Y . If »-0 then 
jg(y_ Y )=0 and we see immediately from (2. 13) that the 
MVLTJE is the MMSLE of r . 

We are now ready to prove the natural random-g ex- 
tension of the Gauss-Markov theorem. We prove the 
theorem by a well-known style like that of Rose [l] 
which ascribes a property that may be called " wide-sense 
sufficiency" to the right-hand side of the normal equa- 
tions for estimating any linear vector function y = C(J of §. 

Theorem- 2.7: The Extended- Gauss-Markov Theorem. 
Given that fr-Xtf^WSiO, %), an estimator y(qXl) 
is the MMSLE based on y of any linear vector function 
Y=Cgof jJif and only if 

(1) y is a M-onbiased estimator of y 
and 

(2) y is a linear vector function 

y — M£ 

of the rift! t 1 1 sid * I v) of the normal equa- 

tions (2.7) where M is qXr. 

Proof: We show that y is the unique MVLTJE of y. 
From Lemma 2.6 it follows that y is the unique M MSLE 
of y. To show the sufficiency of (1) and (2) let 

y* = Ay 

hi »'.'•> 1 ' . 5 f of y. Then 

E(y*) = J5( T ) = £( Y ), 
from which it follows that 

AX V - Cy = MX'S-'Xy. (2.14) 
Sine* (2.14) is an identity in jj we have 

(AX - MX'i-'X) = 0, (2.15) 
(AX - C) =0, (2.16) 
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we can minimize Similarly 

- r ] = var [Mirier - eg] 

- varlAfX'i-lX? + «) - C 
= var[MX'i~HX|i + Xu + 

- var[(MX'Z- ! X - 

= MX'i-'ii-'XM' 


December 


to make 


C V - Cu) 
a + MX'H] 
Mjf'r'M'. 

{using (2.17)) 

Write 
var[y*- y] 

= lA-MX'Z->+MX>Z-mA'-*- l XM>+Z->XM>] 
= [A~MX'i^z{A'-i->XM>} (2.18) 

+2[A- MX'ST 'jXM' + A*X'X ~'XM' 
= non-negative quadratic form + 0+ var [y~ rl . J 
(using (•.'.* 

Since the diagonal elements of a non-negative quadratic 
form are non-negative, 

var[ 7 f - yi\ > var[-y, - *] » - L ■ ■• • , t- 
Thus T is a MVLUE of y- Uniqueness follows from (2.18), 
for if varlY*-T]-var[Y-Y] « « s necessary that 
A =MX'z- ! and heace that y* = f- This establishes the 
sufficiency of conditions (1) and (2). Their necessity fol- 
lows from Lemma 2.6 for (1) and a similar proof by con- 
tradiction for \2). 

2.8 Corollary: The MMSLE of Random §. Given that 
(y - X3) - !FS(0, X) 

then 

b = (X'£~ ! X)"'X'X-> 

is the MMSLE of 3 based on y. 

Proof: Note that (1) E(b-Q) =0 from Lemma 2.4, and 
(2) b~Mg where g<=*&'T£r l y, the right-hand side of the 
normal equations (2.7), and M- (X'S^Xr 1 . The' de- 
[ridtnuJj ) 1 ' * i"l <l r ^ f ^l.ov 


and 


ir[ T * -- - 


(MX'X-'X - C) = 0. 

- var[Ay - Cg] 
«var[i4(*0 +;>-<*] 
= varfAXy + AXu + At 
= var [(AX ~ C)u 4- Aej 
= ASA', (using (2.16)) 


Thee 

In Sects 
type dyn;! 
, ' ' '* 

random 5 


, 2.7, 


n 3 v," fih-.iM review briefly a typical Kalman- 
ni<; fii.dtl .-11 <<: 'h« introduction and 
r r 1 t « r- 1 .*> 1 1 1 - 1 

I ' -i ,„,t , r - n }.< i< \ed b • 
gressi »n theor> e> ei abov< 


3. LINEAR DYNAMIC RECURSIVE EST: \ATiON 

3.1 The Mode! 
A t>pu<il f< rrn t < 15 d ,u< hi t 

, S ',<!*( -< ' - 


ession-lik 

, = T t § t t + 

, = X$ t + 1 


by the r< 
notation) as: 


(3T) 
(3.2) 
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or, in other words, the MMSK for r3,. For a complete dis- 
cussion see [8j. 
The updating equations (3..V; are equivalent to the 


where 


for 1 = 
writt 


The starting equation for §, may be 


3i = j 


(3.4) 


where tfi is a 


of the pre 1 
error*. A> ; 


(3.6) 


y, = (W , y«' , 


,y.')'. 


of V 


more intuitive recursi 
and (2.9)]: 


umed known. All the error vectors are un- 
ween times. Equation (3.1) expresses the 
• (Mr XI) as ;l liIt ' mi ' 1 ir u ' l * ' ri ' ' 
is state g.-t plus a rector w, of random 
i 1 f 1 ,-. ! i dui n - it* 
transition equi? '3.!.i in the parameters J, which is 
lr tt \ .f i ' . - )it r *nd gives the model 
so mud. '»f :t> w^M,t 

3.2 The Recursive Estimation Equation 

In the ei.fei « 4 1 if r ' 1 * -h n\u [5, 6] that the 
MMSLE b,of gj based on all the data sn.yi, • • ■ , y> up 
through time f is given by the recursive updating equa- 
tions: 

6, = + 5ii(-iJK Dr'to - X t b t ,<-t) (8.5a) 

and 

»■ s« = S,,«-» S„. -i« Dr'XAi,-. (3.5b) 
where 

64,U=r ( fc,_»; S !11 ... l =r t S ! _ i T ( '4-Q t ; 1 = 2, ■ • • ,n 
■ D, = R,+XAi,-iX ( ' • • • , 

It is also shown that 


Kalmau. [6] derives these results from the p 
of wide-sense conditional distributions and expectations 
as described, e.g., in [3j. In this approach 6/ is obtained 
as an , rth . t! r>r .11 ti'»n of g, on a linear manifold 
spanned by the elements of y<. As mentioned previously, 
Was is the same as fitting a regression of §, on the ele- 
tt$nts of y, using expected first and second moments in the 
ftpmai equations involved. Result (3.7) represents the 
aptKstal form of orthogonality so obtained, and the fact 
that E(bt -g t )» is mii ir ized ilw» .1 tb same time. 

An t > 1 of d K similar str nger results is 

jjfcfte Bayeaian approach under the Gaussian asaump- 

4&M given in Section 1. The estimator b ( in (3.5a) js 
JjwBly shown to be the posterior mean for g, given y, 
ftms m thus the Bayes estimator under squared error loss 


weighting equations [cf. (2.8) 
= S t [Siu-ib iU -i + x',R: l y t ] (3.8a) 


- [Sr lt -i + X t R 


l X t ] . (3.3b) 
•re convenient titan the 


Usuallv equations (3.5) are r 

v, A hu 1? . tti i ) s .-m j ti- g >-r ><! lh 1 t} " 

common c is, - />< 1 The eq.a 'to >" r »t > ( t > > 
(3.8 follow* from the fallowing two well-known 
lemmas. 

Lemma 3.3: If S= [M~ l + X'R~ l X}- 1 
then 

S = M — MX'D 'XM 

where M is r X r, 1? is p X p, X is p X r, and D » /? + W. 
Proo/: The result follows from showing that 

[M 1 + X'i?- l AT][M - MX'i>- l XM] 
= / - X'0" ! M| X'R-'XM - X'R^XMX'D-'XM 
= / + l-X'D- 1 + X'i? ' - TR-'XMX'D 'jXAf 

- j + jr*->l- so- 1 + / - XMX'Z)- l ]XM 

- / 4- X'/T ! [/ - DD l ]XM = / 
Lemma S.I,: If 6,-SlW- l *.+X f *- , y] 

then 

b,= & 0 + MX'Dr'\y - y] 

where 

Misr Xr, facisrX 1, XispXr, KispXp, ytspXl, 
D=R+XMX', y = X6 0 , and S= [M-' + X'R-'X]- 1 . 
Proqf: Using the previous lemma we see that 
b t = [Af — MX'D"'XM][M 'bo + X'R~ l y] 

= b 0 + MX'[/?"'y - D-'Xbo - D-'XMX'R-'y] 
= b 0 + MX'D~ l \{D - XMX')R~'y - Xb„j 
- bo + MX'D ! [y -y]. 

4. REGRESSION APPROACH DERIVATION 
Using the random-3 regression theory developed in 
Section 2, we derive the distribution of b, - §, with b, ex- 
pressed in the intuitive weighting form (3.8a) and show 
directly that this b, is the MMSLE of & based on all the 
data y, = , y,', • • • , y/ J' up through time <. Since 
equation (3.8a) is equivalent to equation (3.5a) we have 
an indirect derivation of the Kalmau updating optimal 
estimator and its distribution. 

Lemma 4.1: The distribution of b, - ff,. Given the model 
described in equations (3.1) through (3.4), the recursive 
weighting equations (3.8a) and (3.8b) give b, and S, 


(4.1) 


820 

such that 

(b, - ?,) ~ WS(0, S>), t = i, ■ 

Proof: To prove this inductively, suppose the result 
(4. 1 ) is true for time t—1. Then from. 

fc„_,-ff«-r ( 6w-r*w-ii,-r,(bw|-9,-i)-tt« 

we have. 
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Theorem 4.2: The Oplmu 
model described in equatio 
ir- .. i st nating q »ati< 
MMSLE 6, of g, based on 
through time t. 
Proof: Write thi recumv. 


-1 f gi 


rewriting 


Let 


y of b t . (5iHi ' r« r > •- 
(3.1) through (3.4), the re- 
(3.8a) and (3.8b) give the 
he data uj. ft, • ■ ■ , y t up 

ynaniic model (3.1) through 
(4.5) 


■a 


0 » - Tj3i - u 2 . 


(l».„-i - fc)~WS(0,S«i«-i>.! 

Now letting 
represent 

we have, from (4.2) 

^-^^^(4o" i_i y> (4 - 4) 

Using (4,4) and the distribution Lemma 2.4(1) it is seen 
that the recursh eigh ng juafi ns !. 8a) and (3.8b) 
give 6, and S, such that (6,-(J ( )~WS(0, S<). This com- 
pletes the induction step. 

From the initial equations &iia = i«i and S lt o = Qi, the 
premise (4 2 <■ - tisfied it ie i 1 Jlew the Lemma is 
true at { = 1 and therefore by induction at the successive 
times t = 2, 3, - ■ • , n. 

From model (4.4) above, the Extended Gauss-Markov 
Theorem 2.7 tells us that b, is the MMSLE of g, based 
on 6, M .. ,andy t . It remains to be shown that ib, is MMSLE 
} t \ i i t l , ta y • i 1 r i j V( i 

this in the following theorem. 


J?f be var(ef). 

Note that the right hand side of the r 
for (4.5) is 

g* = Xf"R*~ l y* 

which reduces to 


LX.'fir'y. J' 


and that b< is a linear function of £ t *. 

Since b t is a linear function of the right-hand ^ajgjjbf 
the normal equations for (4.5) and b t is u-unbiased from 
Lemma 4.1, we conclude from the Extended Gauss- 
Markov Theorem that b, is MMSLE of g« based on*&8 
data y,* and therefore y s . This then establishes the opti- 
mally of b, from the point of view of regressing y,* on Xf. 

We have shown that bt is also the MMSLE of based 
on biH~\ and y,. In addition to the statistic g,*, the sta- 
tistics b, , .i and y, may well be called wide-sense suffi- 
cient statistics for estimating any linear function y=C§ 
of § just as X'X-'y is the complete and sufficient statistic 
for the fixed-? model y = X§ -ft where e~ N(0, £). 

We show the final property (3.7) in the following lemma. 

Lemma 4.3: Given the conditions of Theorem 4.2, 

E(b t - fy)yl 0. (4.6) 

Proof: Using Lemma 2.4(3) in model (4.5) 
E{b, - h)y*' = 0 
and (4.6) follows from this. 

5. CONCLUDING REMARKS 

The conventional regressionist will have found our ap- 
proach to the Kalman recursive results in Section 4 more 
natural since we derived thmi from the regression point 
of view. One <>. the keys to our approach is the fact that 
the recursive dynamic model (3.1) to (3.4) may be written 
in the form yt = X + t t *. (4 ">« \V< p in to show u 
future articles that this opens the way for the application 


... to 


rohler 


where the thl • ' i ition. for example, ,omes from 


«ne c-tim t»m ^ tli ! nr. >. -t > «i e the R, 
and 0, variance matrices. 

Before clos * 1 I 1 that KalnWs recur- 

sive estimatic th »r\ is con* ' » r < general than 
the form discussed in. this article. The more general prob- 
lem is to find an optimal estimator b t based on all the 
data y, up through time s: If $<t, s = L or »>t, this is. 


sqd?: ; a. f •? </••> -• <-. :i' . i 

For brevity we have dealt ^ 

iraj.< it, c. -i >f 'ilif i , >*i: 

[ft*< .• ' \ < > * - 
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